SalGAN: Visual Saliency Prediction with Adversarial Networks
نویسندگان
چکیده
Visual saliency prediction in computer vision aims at estimating the locations in an image that attract the attention of humans. A saliency map is a heatmap that represents the probability of each corresponding pixel in the image to capture human attention. These saliency maps have been used as soft-attention guides for other computer vision tasks, and also directly for user studies in fields like marketing. This paper explores adversarial training [2] for visual saliency prediction. The discriminator distinguishes between samples from the true data distribution and samples produced by the generator. In our case, this data distribution corresponds to pairs of real images and their corresponding visual saliency maps. We show how adversarial training significantly benefits a wide range of visual saliency metrics, without needing to specify a tailored loss function. Our results achieve stateof-the-art performance with a simple deep convolutional network whose parameters are refined with a discriminator.
منابع مشابه
SalGAN: Visual Saliency Prediction with Generative Adversarial Networks
We introduce SalGAN, a deep convolutional neural network for visual saliency prediction trained with adversarial examples. The first stage of the network consists of a generator model whose weights are learned by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency maps. The resulting prediction is processed by a discriminator network traine...
متن کاملTask Specific Visual Saliency Prediction with Memory Augmented Conditional Generative Adversarial Networks
Visual saliency patterns are the result of a variety of factors aside from the image being parsed, however existing approaches have ignored these. To address this limitation, we propose a novel saliency estimation model which leverages the semantic modelling power of conditional generative adversarial networks together with memory architectures which capture the subject’s behavioural patterns a...
متن کاملASP: A Fast Adversarial Attack Example Generation Framework based on Adversarial Saliency Prediction
With the excellent accuracy and feasibility, the Neural Networks (NNs) have been widely applied into the novel intelligent applications and systems. However, with the appearance of the Adversarial Attack, the NN based system performance becomes extremely vulnerable: the image classification results can be arbitrarily misled by the adversarial examples, which are crafted images with human unperc...
متن کاملCompressed-Sampling-Based Image Saliency Detection in the Wavelet Domain
When watching natural scenes, an overwhelming amount of information is delivered to the Human Visual System (HVS). The optic nerve is estimated to receive around 108 bits of information a second. This large amount of information can’t be processed right away through our neural system. Visual attention mechanism enables HVS to spend neural resources efficiently, only on the selected parts of the...
متن کاملVisible Progress on Adversarial Images and a New Saliency Map
Many machine learning classifiers are vulnerable to adversarial perturbations. An adversarial perturbation modifies an input to change a classifier’s prediction without causing the input to seem substantially different to human perception. We deploy three methods to detect adversarial images. Adversaries trying to bypass our detectors must make the adversarial image less pathological or they wi...
متن کامل